Papers with distributional representations
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks (D19-60)
Copied to clipboard
| Challenge: | Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations. |
| Approach: | They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention. |
| Outcome: | The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets. |
From Brain Space to Distributional Space: The Perilous Journeys of fMRI Decoding (P19-2)
Copied to clipboard
| Challenge: | Recent work in cognitive neuroscience has introduced models for predicting distributional word meaning representations from brain imaging data. |
| Approach: | They propose to use several alternative measures to evaluate the predicted distributional space against a corpus-derived distributional spatial space. |
| Outcome: | The proposed model performs poorly on the most common metrics, while still delivering promising results. |
Modeling Semantic Plausibility by Injecting World Knowledge (N18-2)
Copied to clipboard
| Challenge: | Existing models for semantic plausibility are based on distributional data, but injecting knowledge about entity properties provides a substantial performance boost. |
| Approach: | They propose to inject manually elicited knowledge about entity properties into a dataset to improve plausibility models. |
| Outcome: | The proposed dataset is a great testbed for semantic plausibility models . it shows that injection of knowledge about entity properties improves performance . |
Cross-Topic Distributional Semantic Representations Via Unsupervised Mappings (N19-1)
Copied to clipboard
| Challenge: | Existing distributional semantic models cannot capture the distinct meanings of polysemous words, resulting in conflated word representations of diverse contextual semantics. |
| Approach: | They propose a distributional semantic model that learns multiple representations of a word based on different topics. |
| Outcome: | The proposed model outperforms single-prototype models on NLP downstream tasks. |
Short-Term Meaning Shift: A Distributional Exploration (N19-1)
Copied to clipboard
| Challenge: | a new study examines the phenomenon of short-term meaning shift in online communities . the authors use distributional representations to explore the phenomenon . |
| Approach: | They propose to use distributional representations to explore short-term meaning shift in online communities. |
| Outcome: | The proposed model has problems distinguishing meaning shift from referential phenomena, and measures contextual variability to remedy this. |
Are Word Embeddings Really a Bad Fit for the Estimation of Thematic Fit? (2020.lrec-1)
Copied to clipboard
| Challenge: | In recent years, vectors derived from neural network training have replaced count-based distributional semantic models as a de facto standard for word representation in NLP. |
| Approach: | They propose to evaluate count models and word embeddings on thematic fit estimation by taking into account a larger number of parameters and verb roles and introducing dependency-based embedders in the comparison. |
| Outcome: | The proposed model outperforms count models and word embeddings in thematic fit estimation tasks while introducing dependency-based embedders. |
Undersampling Improves Hypernymy Prototypicality Learning (L18-1)
Copied to clipboard
| Challenge: | supervised hypernymy detection suffers from overfitting hypernies in training data. |
| Approach: | They propose a method that can alleviate the problem of overfitting hypernyms in training data by using distributional representations for unknown word pairs. |
| Outcome: | The proposed method alleviates the problem of overfitting hypernyms in training data and improves distributional prototypicality learning for unknown word pairs. |
SWEAT: Scoring Polarization of Topics across Different Corpora (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using two additional wordsets, we compute the relative polarization of a topical wordsetting across two distributional representations. |
| Approach: | They propose a new measure to compute the relative polarization of a topical wordset across two distributional representations using two additional wordsetes deemed to have opposite valence to represent two different poles. |
| Outcome: | The proposed measure is validated by a case study and validated in a randomized controlled trial. |